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ABSTRACT 



This study focuses on programmatic factors that predict 
retention for individuals in drug and alcohol treatment programs through 
secondary analysis of data from the National Treatment Improvement Evaluation 
Study (NTIES) . It addresses the relationships between completion rates, 
lengths of stay, and treatment modality. It examines the effect of 
program- level factors on treatment retention. NTIES intake and treatment 
episode completion data for 3,117 clients across 61 service delivery units 
were used in the study. Data were organized into eight analysis cells defined 
by treatment modality and intended length of stay. Results show that, in 
general, shorter programs have higher treatment episode completion rates but 
longer programs retain clients for longer periods. Long-term residential 
programs exhibit higher treatment episode completion rates than long-term 
outpatient or correctional programs. The median length of stay for clients in 
long-term or correctional programs with intended lengths of treatment of 120 
days or more were lower than in the 90-119 day programs, suggesting that 
setting the intended length of treatment at 120 days or more may be 
counterproductive with respect to retaining clients. Implications for 
research, policy, and practice are discussed. An appendix provides a 
description of the National Treatment Improvement Evaluation Study and Center 
for Substance Abuse Treatment Demonstrations (1990-1992) . (Contains 12 tables 
and 24 references.) (MKA) 
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Foreword 



CSAT works to improve the lives of those affected by alcohol and other substance abuse, 
and, through treatment, to reduce the ill effects of substance abuse on individuals, families, 
communities, and society at large. Thus, one important mission of the Center for Substance 
Abuse Treatment (CSAT) is to expand the knowledge about and the availability of effective 
substance abuse treatment and recovery services. To aid in accomplishing that mission, CSAT 
has invested and continues to invest significant resources in the development and acquisition of 
high quality data about substance abuse treatment services, clients, and outcomes. Sound 
scientific analysis of this data provides evidence upon which to base answers to questions about 
what kinds of treatment are most effective for what groups of clients, and about which treatment 
approaches are cost-effective methods for curbing addiction and addiction-related behaviors. 

In support of these efforts, the Program Evaluation Branch (PEB) of CSAT established 
the National Evaluation Data Services (NEDS) contract to provide a wide array of data 
management and scientific support services across various programmic and evaluation activities 
and to mine existing data whose potential has not been fully explored. Essentially, NEDS is a 
pioneering effort for CSAT in that the Center previously had no mechanism established to pull 
together databases for broad analytic purposes or to house databases produced under a wide array 
of activities. One of the specific objectives of the NEDS project is to provide CSAT with a 
flexible analytic capability to use existing data to address policy-relevant questions about 
substance abuse treatment. This report has been produced in pursuit of that objective. 

Participant retention in alcohol and other substance abuse treatment programs is an 
important outcome that has been found to be a mediator of subsequent client-level outcomes such 
as reduced alcohol and other drug use, residential stability, and economic security. The analysis 
reported here focused on programmatic factors that predict retention through a secondary analysis 
of data from the National Treatment Improvement Evaluation Study (NTIES). (The appendix to 
this report contains a summary description of NTIES.) For policy planners and treatment 
providers, results of the analyses show that service delivery unit (SDU)-level factors, such as 
whether the client reported seeing their treatment plan, providing vocational training, the use of 
staff specifically designated as case managers, and providing services tailored to populations 
defined by specific characteristics (e.g., homeless, pregnant, etc.) clearly increase the odds of 
program completion, after controlling for client-level factors. In addition, lower 
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retention rates for programs with intended durations of 120 days or more implies shorter (e.g., 90 
days), more intensive programs may be more successful. 

Sharon Bishop 
Project Director 

National Evaluation Data Services 
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Executive Summary 



1. INTRODUCTION 

Participant retention in alcohol and other drug abuse treatment programs is an important 
outcome to policy makers, program administrators, and program staff and has been found to be a 
mediator of subsequent client-level outcomes such as reduced alcohol and other drug use, 
housing stability, and economic security gains post-treatment. Reviews of the treatment retention 
literature reveal that investigators have most often focused on identifying participant 
characteristics that predict retention (De Leon, 1991; Moos, Finney, & Cronkite, 1990; Stabler, 
Cohen, Shipley, «& Bartlet, 1993). However, the literature has shown the predictive power of 
participant characteristics to be relatively weak and inconsistent (DeLeon, 1991; Stabler et al., 
1993; Kleinman et al., 1992). 

The present study focused on programmatic factors that predict retention through a 
secondary analysis of data from the National Treatment Improvement Evaluation Study (NTIES). 
The study addressed the relationships between completion rates, lengths of stay, and treatment 
modality and examined the effect of program-level factors on treatment retention. A summary 
description of the study is provided in the Appendix to the report. 

2. METHODS 

NTIES intake and treatment episode completion status data for 3,1 17 clients across 61 
SDUs (services delivery units) were used in the study. The data were organized into eight 
analysis cells defined by treatment modality (non-methadone outpatient, correctional, short-term 
residential, long-term residential) and intended length of stay (21-30 days, 41-89 days, 90-1 19 
days, 120 days or more) so that subsets of clients that might differ significantly in terms of 
treatment program type could be considered separately. 

Three types of contrasts were chosen: 

■ Comparisons within modality across intended length of treatment strata 

■ Cross-modality comparisons within intended length of treatment strata 

■ Comparison of short-term residential (21 to 30 days of intended treatment) to long- 
term residential (120 or more days of intended treatment). 
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For each of the eight analysis cells, chi-square tests of independent proportions were carried out, 
comparing treatment episode completion rates across cells for pre-specified contrasts. Mean 
LOS values for each cell were then compared across cells for the same contrasts. Logistics 
regression methods were used to investigate which factors predict completion of treatment within 
each of the eight analysis cells. 

3. RESULTS 

The descriptive comparisons of completion rates and median lengths of stay for the eight 
analysis cells revealed that: 

■ In general, shorter programs have higher treatment episode completion rates but 
longer programs retain clients for longer periods 

■ Long-term residential programs exhibit higher treatment episode completion rates 
than long-term outpatient (non-methadone) or correctional programs 

■ The median length of stay for clients in long-term outpatient (non-methadone) or 
correctional programs with intended lengths of treatment of 120 days or more were 
lower than in the 90-119 day programs, suggesting that setting the intended length of 
treatment at 120 days or more may be counterproductive with respect to retaining 
clients. 

The results of the logistic regression analyses showed that SDU-level factors, such as whether the 
client reported seeing their treatment plan, providing vocational training, the use of staff 
specifically designated as case managers, and providing services tailored to populations defined 
by specific characteristics (e.g., homeless, pregnant, etc.) clearly increase the odds of program 
completion, after controlling for client-level factors. Whether the client reported seeing their 
treatment plan is by far the most predominant predictor of treatment completion, found to be 
significant in all of the models in which it was included. The predominance of this effect across 
analysis cells suggests that engaging the client in the treatment plan development process may be 
an important determinant of treatment completion. 
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4. IMPLICATIONS FOR RESEARCH, POLICY, AND PRACTICE 

Possibilities for further analyses using the NTIES data set include: 

■ An investigation of interactions between client level variables and SDU-level 
variables 

■ A closer look at those clients who drop out in order to develop profiles of “early” and 
“late” leavers 

■ More focus on why clients leave programs 

■ Investigation of patterns in terms of when people leave their programs 

■ Investigations that relate treatment episode completion and treatment outcomes 
(reduced substance abuse, etc.). 

Implications for policy that can be drawn from these results are: 

■ Consideration should be given to developing an intensive, moderate-length (about 90 
days) residential program since our findings show lower retention and completion for 
excessively long intended treatment (120 days or more). 

■ Case management can affect retention and completion but needs further study. 

An implication for treatment practice that can be drawn from these results is that showing a client 
his or her treatment plan can have a positive effect on completion and retention. 
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I. Introduction 



1. STATEMENT OF PROBLEM 

“How important is the matching of client services to perceived client need on retention in 
substance abuse treatment?” was one of the sample questions asked by CSAT in the original 
Statement of Work for the NEDS contract. In fact, matching is but one of many factors of 
interest when assessing what influences retention. Participant retention in alcohol and other drug 
abuse treatment programs is an important outcome to policy makers, program administrators, and 
program staff and has been found to be a mediator of subsequent client-level outcomes such as 
reduced alcohol and other drug use, residential stability, and economic security. To the degree 
these outcomes will be attained more successfully if participants stay in their treatment programs 
(as well as longer-term community-level outcomes such as reduced utilization of expensive and 
inappropriate services, lowered crime rates, higher productivity, etc.), then further study of the 
relationship between retention and treatment is clearly warranted. And numerous studies have in 
fact reported an association between retention in treatment and client outcome (Baekeland & 
Lundwall, 1975; De Leon, 1991; Hser, Angler, & Liu, 1991; McKusker, Stoddard, Frost, & Zorn, 
1996; Stabler et al., 1993; Stark & Campbell, 1988). 

However, the relationship between retention and successful outcomes can often reflect 
self-selection factors rather than (or in addition to) the treatment experience (De Leon, 1991 ; 
Moos et al., 1990). Reviewers of the treatment retention literature note that investigators have 
most often focused on identifying participant characteristics that predict retention (De Leon, 

1991; Moos et al., 1990; Stabler et al., 1993). In part, this is because analysts can use participant 
characteristics to (in theory at least) “control for” the self-selection problem (see, for example. 
Moos et al., 1990). There is also a more practical reason: participant characteristics are typically 
easier to measure than potentially important programmatic predictors such as, for example, 
treatment milieu. In any case, the literature has shown the predictive power of participant 
characteristics to be relatively wetik and inconsistent (DeLeon, 1991; Stabler et al., 1993; 
Kleinman et al., 1992). One particularly stark example, an article in the 1991 volume of the 
International Journal of the Addictions, found that participants with a history of depression were 
significantly more likely to stay in treatment (Agosti, Nunes, Stewart, & Quitkin, 1991). This is 
remarkable only because an article in a subsequent issue of the same volume found that 
participants with depression histories were less likely to stay in treatment (Williams & Roberts, 
1991). Similar inconsistencies were found across sites in two multi-site evaluations which 
examined retention of homeless persons in treatment (Orwin et al., 1994; R.O.W. Sciences, 

1995). Orwin et al. (1994) identified several potential reasons; these included differences in 
target populations, differences in the interventions, differences in retention policies, and 
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differences in the definition of dropout. The only characteristic that seems to predict retention 
with any consistency is whether treatment was court mandated as a condition of avoiding prison 
(De Leon, 1991; Gallant, Bishop, & Faulkner, 1968; Rosenberg & Liftik, 1976; Stark & 
Campbell, 1988) or receiving public assistance (Brizer, Maslansky, & Galanter, 1990). Court- 
mandated participants are typically retained at higher rates, but often only over the short run (e.g., 
< 30 days). 

As noted by Stabler et al. (1993), if the implicit goal of modeling participant 
characteristics is to better understand how to retain clients in treatment programs, then an 
alternative strategy for achieving this goal is to learn why they left, i.e., what precipitated their 
exit. De Leon (1991) has also noted that client reasons for dropout offer considerations for how 
programs could modify their service delivery to improve retention, and are therefore responsive 
to programmatic needs. Lewis and Ross (1984) further emphasize this point, and add that there 
should be more focus on the “when” of dropout as patterns in the timing of this event may 
provide information programs could use to develop and target strategies that engage clients 
during critical periods. In a similar vein. Stark and Campbell (1988) argued that retention 
researchers must be more responsive to the needs of treatment providers and their clients to 
recognize, cope with, and better understand the conditions that precipitate dropout. They went 
on to suggest that with sufficient information from researchers, dropout prevention — like relapse 
prevention — might profitably become a standard operating procedure in the treatment of • 
substance abuse. Orwin et al. (1998) emphasized a similar theme in their retention analyses of 
interventions from the 14-site NIAAA Cooperative Agreement program for homeless substance 
abusers. That study focused on (1) service components related to retention, (2) the importance of 
attending to phase transitions in preventing dropout, and (3) the importance of being 
programmatically responsive to the reasons clients leave prior to completion. 

2. THE PRESENT STUDY 

The present study also focuses on programmatic factors. It presents the results of a 
secondary analysis of data from the National Treatment Improvement Evaluation Study (NTIES). 
The NTIES project was a five-year longitudinal study of the impact of substance abuse treatment 
on a total of 5,388 clients purposively sampled from public substance abuse treatment programs 
(service delivery units or SDUs) that were funded by the Center for Substance Abuse Treatment 
(CSAT)'. (A summary description of the NTIES study is contained in the Appendix to this 
report.) The NTIES data set affords an excellent opportunity to examine the effects of 



' CSAT defines an SDU as a single treatment modality provided at a single site. 
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programmatic influences on retention because of the large array of SDU-level variables that were 
collected from program directors in the NTIES Baseline Administrative Report (NEAR). This 
analysis will augment the analyses from the original NTIES report (NORC, 1997) by developing 
more comprehensive prediction models that utilize both SDU-level and client-level variables. 
Specifically, it will use the client-level variables to control for the effects of participant 
characteristics (case mix) and make greater use of the wide range of SDU-level variables 
available from the NBAR to identify programmatic influences. Because client-level variables 
have been the focus of so much study, new gains in knowledge about best practices for increasing 
retention in drug treatment may come from a greater focus on SDU-level. characteristics. 

3. STUDY QUESTIONS 

This study addressed the following specific questions: 

■ How do treatment episode completion rates and actual lengths of stay compare for 
SDUs with different intended lengths of stay within the same modality? 

■ How do treatment episode completion rates and actual lengths of stay compare for 
SDUs from different modalities within the same intended length of stay? 

■ What program-level factors appear to improve retention, and by how much? 

■ To what degree does the inclusion of program-level factors improve overall model fit 
over and above client-level factors alone? 

■ How do the above relationships vary by modality and intended length of stay? 

■ What are the implications for future research and the treatment community? 
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This section describes the selection of analysis cells and contrasts, the final sample, and 
the statistical techniques used in the analysis. 

1. SELECTION OF ANALYSIS CELLS AND CONTRASTS 

The first step in the analysis involved organizing the NTIES data in a way that would 
allow those subsets of clients that might differ significantly in terms of treatment program type 
(treatment modality and intended length of treatment) to be considered separately. Clients who 
participated in treatment at SDUs that provided methadone maintenance programs were excluded 
from this analysis because (1) the intended duration of treatment in such programs is generally an 
indefinite period of time and (2) there is no completion event to analyze. Exhibit II- 1 shows 
treatment modality stratified by intended length of treatment in days. Cells containing patients 
from only one SDU were excluded in order to avoid the undue influence of a single SDU on the 
results. Two intended length of treatment strata (41-60 days and 61-89 days) for correctional 
SDUs were combined because the patients in each cell represented only one SDU and no other 
modality reported either of these time intervals. A total of 8 cells were considered in this 
analysis. These cells included: 90 to 119 days non-methadone outpatient, 120 days or more non- 
methadone outpatient, 21 to 30 days short-term residential, 120 days or more long-term 
residential, 21 to 30 days correctional, 41 to 89 days correctional, 90 to 119 days correctional, 
and 120 or more days correctional. 

Three types of contrasts were chosen for this analysis. The first type of contrast consisted 
of comparisons within modality across intended length of treatment strata and included: 90 to 

119 day non-methadone outpatient versus 120 days or more non-methadone outpatient; and 21 to 
30 days correctional versus 41 to 89 days correctional versus 90 to 1 19 days correctional versus 

1 20 or more days correctional. The second type of contrast consisted of cross-modality 
comparisons within intended length of treatment strata and included: 90 to 119 days outpatient 
versus 90 to 1 19 correctional; 120 or more days outpatient versus 120 or more days long-term 
residential versus 120 or more days correctional; and 21 to 30 days short-term residential versus 
21 to 30 days correctional. The final type of contrast consisted of a comparison of short-term 
residential (21 to 30 days of intended treatment) to long-term residential (120 or more days of 
intended treatment). 
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2 . DESCRIPTION OF SAMPLE FOR ANALYSIS 

Both intake (NRIQ) and follow-up data (NTIES Post-discharge Assessment 
Questionnaire — ^NPAQ) are available for a total of 4,41 1 clients in the NTIES data set. Only 61 
of the 71 SDUs from which the 4,41 1 participants were drawn have Baseline Administrative 
Forms (NBAR). This subset was used as a starting point because information on treatment 
completion status (TXCOMPL) was available only for these clients.^ 

Several exclusions were made from this initial subset of clients. Data on a total of 3,1 1 7 
clients were available for analysis after the following exclusions were made. Those clients for 
whom NBAR data were unavailable were excluded because intended length of treatment (B242) 
and treatment modality (CELL2) are key SDU-level variables in this analysis. Limiting the 
analysis to these clients reduces the sample size to 3,960. Further limiting the analysis to those 
clients drawn from SDUs that provided information on intended length of treatment reduces the 
sample size to 3,549. In restricting the analysis to the 8 cells previously mentioned, the 
following clients were excluded: 16 non-methadone outpatient clients from the 31-40 day 
stratum; 382 short-term residential clients (213 in the 6-11 day stratum and 169 in the 120 or 
more days stratum); and 34 clients in the long-term residential 90-1 19 day stratum (total 
excluded=432). 

3. STATISTICAL ANALYSIS 

3.1 Descriptive Comparisons 

For each of the 8 analysis cells, descriptive statistics were computed for the variables 
treatment completion status (TXCOMPL) and length of stay (LOS) . Chi-square tests of 
independent proportions were carried out, comparing completion rates across cells for pre- 
specified contrasts. Mean LOS values for each cell were then compared across cells for the same 
contrasts. 

3.2 Construction of Prediction Models 

Logistic regression methods were used to investigate which factors predict completion of 
treatment within each of the eight analysis cells. The models used for predicting treatment 
completion status were developed using SAS® PROC LOGISTIC DESCENDING (SAS 



- TXCOMPL was constructed by NORC based on multiple data sources and best available evidence on each client. 
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Institute, 1990). The LOGISTIC procedure fitted the multiple logistic regression models to the 
binary dependent variable TXCOMPL (l=treatment completed, 0=treatment not completed) 
using the maximum likelihood method. Maximum likelihood estimation gives the unique set of 
estimates of all possible sets of values that could describe the population from which the sample 
came and is most likely to give rise to the observed data. 

The regression coefficient in a logistic regression model indicates the effect of a variable 
on the log odds of the outcome when all other variables in the model are held constant. The 
coefficient shows the magnitude of the increase or decrease in the log odds produced by one unit 
of change in the value of the independent variable whose coefficient is being observed. To 
obtain the odds ratio for a binary coded variable, the following formula is used: e'’. In all logistic 
models constructed in this analysis, the probability that the dependent variable, treatment 
completed, =1 was tested. 

Exhibit II-2 describes the client-level predictors that were entered into each of the logistic 
regression models. These background variables are very similar to those used by NORC in their 
multiple regression outcome analyses. Each of these variables had shown a significant bivariate 
relationship with one or more of the treatment outcomes (post-discharge drug use, alcohol use, 
crime activities, psychiatric problems, medical problems and employment problems severity 
scales) considered by NORC in their analyses. 

SDU-level independent variables were then selected for inclusion in the logistic 
regression models. In the first step of this selection process, we carried out exploratory data 
analysis of the NEAR variables in order to determine the number of missing values and degree of 
variation exhibited by these variables within each analysis cell. Several exclusions were made 
after reviewing the results of this exploratory analysis. NEAR variables missing 30 percent or 
more of their values were excluded from analysis as were variables exhibiting no variation. 

More than 35 percent of the values for one NEAR variable of interest, “formal treatment 
plan developed,” were missing for five of the eight analysis cells and this variable exhibited no 
variation in the remaining three analysis cells. The treatment plan is considered an essential 
component of substance abuse treatment and the development and periodic update of such a plan, 
with the knowledge of the individual client, are generally required for funding (Hubbard, et al. 
1989). In order to include some measure of the existence of a treatment plan, a client-level 
variable from the NTIES Treatment Exit Questionnaire (NTEQ), “patient saw treatment plan,” 
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EXHTOIT n-2 

Dependent and Background Variables Used in 
Logistic Regression Models 


DEPENDENT VARIABLE 


Description 


Data Source(s): 


Treatment Completion Status 


Split from Treatment (TXCOMPL - Analysis file) 


background variables 


Description 


Data Source(s): 


Age 


Age from Birth Date (R34 - NRIQ) 


Race/Ethnicity 


Race (R44 - NRIQ) 


Hispanic 


Hispanic Origin (R43 - NRIQ) 
Languages Spoken (R47 - NRIQ) 
Language Preferred (R50 - NRIQ) 


Gender 


Gender (R29 - NRIQ) 


Years of Education 


Highest Grade Attended (R58 - NRIQ) 


Prior Treatment Episode 


Prior Alcohol Treatment (R255 - NRIQ) 
Prior Drug Treatment (R256 - NRIQ) 


Legal Pressure to Seek Treatment 


Drug Treatment Required (R373 - NRIQ) 


Pre-treatment Drug Use Severity Scale 


RDRUSEV - Analysis File 


Pre-treatment Alcohol Use Severity Scale 


RALCSEV - Analysis File 


Pre-treatment Crime Activities Severity Scale 


RCRMSEV - Analysis File 


Pre-treatment Psychiatric Problems Severity Scale 


RPSYSEV - Analysis File 


Pre-treatment Medical Problems Severity Scale 


RMED SEV - Analysis File 


Pre-treatment Employment Problems Severity Scale 


REMPSEV - Analysis File 


Primary Drug 


In Treatment for ” (R1 13M1 through M13 - NRIQ) 



was considered in the independent variable screening process. The wording of the NTEQ 
question (T1 53) is “while you were at the program, did you see a treatment plan or a list of 
treatment goals?” The text that precedes this item describes what might constitute a plan but 
does not distinguish whether the plan was formalized or not (a plan ‘can include things such as 
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report of whether he/she saw a treatment plan is not a proxy for the formal treatment plan item, it 
can serve as an indication of the priority the SDU placed on treatment planning during the 
client’s stay and thus as a program-level variable that may distinguish completers from non- 
completers. 

After applying the exclusion criteria, the remaining NEAR variables were analyzed by 
treatment completion status. In this bivariate analysis, data from categorical NEAR variables 
were cross-tabulated and chi-square test statistics were used to investigate relationships between 
variables. Pearson correlation coefficients were used to investigate relationships between data 
from continuous NEAR variables and treatment completion status. This bivariate analysis was 
carried out within each of the analysis cells. Variables found to be significantly associated with 
treatment completion status within an analysis cell were then added to the logistic regression 
model for that cell. In order to investigate whether this bivariate screening process was 
excluding independent variables that might be related to the dependent variable in a multivariate 
context (suppression effects), SDU-level variables that were correlated with TXCOMPL in any 
one of the analysis cells were included in the logistic regression models of all cells. 

After checking for suppression effects, a sensitivity analysis was conducted as a way to 
assess the “robustness” of the results. This sensitivity analysis consisted of recomputing the 
denominator of non-completers used in the regression models. In order to do this, clients in all 8 
analysis cells were categorized in terms of discharge status using the NPRF item ‘reason for 
discharge’ (F18A) and the NTEQ item ‘patient’s reasons for ending treatment’ (T233A1-A3). 
Clients’ reasons for ending treatment were used to categorize some of the data that were labeled 
missing for the NPRF reason for discharge variable. In the sensitivity analysis, only those clients 
who could be classified as having had definite negative exits from treatment (did not complete 
treatment by patient’s choice, did not complete treatment by administrative choice, did not 
complete treatment unclear by whose choice) were included as non-completers in the within-cell 
logistic regression models. 

After the SDU-level variables were added to the regression models, the improvements in 
model fits were assessed using several goodness-of-fit measures. The first of these measures is 
often called “model improvement” and uses the -2LL (-2 times the log of the likelihood) of the 
model. It is customary to use -2 times the log of the likelihood since the likelihood is a small 
number less than 1 . A good model is one that results in a high likelihood of the observed results 
which translates into a small value for -ILL. “Model improvement” represents the change in - 
ILL between successive steps of building the model and it tests the null hypothesis that the 
coefficients for the variables added at the last step are 0. This test is comparable to the F-change 
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test in multiple regression and the degrees of freedom are the difference between the number of 
parameters in the two models being compared. In this analysis, the -ILL values of regression 
models containing only the background variables are compared to the -ILL values obtained from 
models that include both the background variables and the SDU-level variables. 

The second goodness-of-fit measure uses classification table comparisons of predictions 
to observed outcomes. Specifically, these measures include percent correct, sensitivity, 
specificity, false positive rate, and false negative rate. Percent correct is the number of outcomes 
that were correctly predicted by the model. Sensitivity is the ratio of the number of correctly 
classified events (completers) divided by the total number of events. Specificity is the ratio of 
the number of correctly classified nonevents divided by the total number of nonevents (non- 
completers). 

The third goodness-of-fit measure, the adjusted coefficient of determination, uses the 
formula =R'/R'„ax (where R'= 1 - [L(0)/1(P)]''" and R'„,, = 1 - [L(0)]''" ) and is comparable 
to the coefficient of determination used in general linear models. The final measure used to 
assess model fit is the percentage of concordant observations. This measure represents the 
proportion of the total observation pairs (all pairs of observations comprised of one completer 
and one non-completer) in which the completer has a higher probability of being classified as 
such than the non-completer. A model with a higher proportion of concordant observation pairs 
has better predictive ability than a model with a lower proportion of such pairs.^ 



^ We had originally proposed using survival and hazard analysis techniques to analyze length of stay. The rationale was 
that dichotomization of the dependent variable (e.g., completion versus non-completion) wastes information by 
ignoring the continuous nature of the time dimension, that is, how long do participants stay before they exit and what 
participant characteristics affect their length-of-stay. Sometimes called failure-time models, these techniques have 
been used in the evaluation of drug treatment programs when the outcome of interest is time-based (e.g., time to 
relapse, time to arrest) (Fisher and Anglin, 1987). They are applicable in retention analyses where the outcome is time 
to dropout if the programs are of fixed length and program completion can be conceptualized as the absence of non- 
completion during the intervention period. In this case, program completers are censored cases, because the analyst 
cannot know how long the client would have stayed in program had the intervention period been longer. These 
conditions have applied in other analyses of length of stay (e.g., Orwin et al., 1998), but did not apply here for three 
reasons. First, interventions were not of fixed length. In some programs defined as long-term, for example, some 
clients still completed their programs in as little as 3 days. Second, program completion was not the absence of a non- 
completion event, but often the presence of a completion event (e.g., follows treatment plan, remains clean for a certain 
length of time, etc.). Third, the LOS distribution of completers and non-completers were highly overlapping, indicating 
that completion events routinely occurred long before non-completion events. Under these conditions, it would have 
been inappropriate to define completers as censored, and equally inappropriate to apply survival and hazard analysis to 
the LOS data. Consequently, these plans were dropped and the multivariate analyses were focused on program 
completion rather than LOS. 
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III. Results 



This chapter presents the results of descriptive comparisons across analysis cells and 
within-cell logistic regressions. 

1. DESCRIPTIVE COMPARISONS 

Exhibit III-l presents reasons for discharge across the 8 cells of interest. As shown, the 
distributions of the reasons for discharge from the NTIES treatment episode varied across 
modalities. The 120 days or more non-methadone outpatient and long-term residential cells 
exhibited the highest percentages of clients whose reasons for discharge were attributed to patient 
choice. The correctional analysis cell in this stratum exhibited the highest percentage of clients 
whose reasons for discharge were attributed to administrative choice. 

Exhibit III-2 shows the sample size, number and proportion of clients who completed 
treatment, median LOS, and LOS range for each of the 8 cells considered in the analysis. Several 
patterns are apparent in these descriptive data. First, completion rates are lowest for the cells in 
the longest intended length of treatment stratum across all four modalities. Second, completion 
rates in the correctional modality tend to decrease as length of intended treatment increases. 
Finally, the median lengths of stay and the completion rates for the 120 days or more non- 
methadone outpatient cell and the 120 days or more correctional cell are lower than the median 
lengths of stay and the completion rates for the cells in the corresponding modality in the 90-1 19 
day intended length of treatment stratum. A comparison of the LOS ranges shows a high degree 
of overlap for all of the cells considered in the analysis. 

Exhibit III-3 presents the results of descriptive comparisons across analysis cells for the 
pre-specified contrasts. For all of the comparisons of completion rates within modality and 
across intended length of treatment strata, analysis cells with shorter intended lengths of 
treatment had significantly higher completion rates (p<0.001) than cells with longer intended 
lengths of treatment. In terms of mean LOS, only the correctional cells differed significantly 
from each other. 

Only one of the three cross-modality comparisons of completion rates within intended 
length of treatment strata yielded statistically significant results. Those analysis cells in the 120 
days or more intended length of treatment stratum differed significantly from each other in terms 
of completion rates, with the long-term residential cell exhibiting the highest rate. The mean 
lengths of stay for the non-methadone outpatient cell was significantly higher than the long-term 
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Exhtoit in-3 

Descriptive Comparisons of Treatment Completion Rates 

Within Analysis Cells 


contrasts 


RESULTS 


Completion Rates 


Mean Length of Stay 


Non-methadone outpatient with 90-1 19 days of 
intended treatment versus non-methadone outpatient 
with 120 or more days of intended treatment 


42.4% versus 27.8% 

with 1 DF = 24.95 
/KO.OOl 


115.5 days versus 1 2 1 .2 days 
t with 792 DF = -1.00, n.s. 


Correctional with 21-30 days of intended treatment 
versus correctional with 41-89 days of intended 
treatment versus correctional with 90-119 days of 
intended treatment versus correctional with 120 or 
more days of intended treatment 


88.9% versus 94.5% versus 
51.6% versus 22% 

Spearman Rank Correlation 

r-rauks = -0.60 
p<0.001 


28 days versus 67.8 days 
t with 342 DF =-10.94 p<0.0001 

67.8 days versus 1 10.5 days 
t with 249 DF =-5.28 p<0.000\ 

1 10.5 days versus 93.4 days 

t with 363 DF = 2.32 p<0.02 

28 days versus 1 10.5 days 
/with 339 DF =-15.91 p<0.0001 

28 days versus 93.4 days 
/with 456 DF =-15.36 p<0.000\ 

67.8 days versus 93.4 days 
/with 366 DF=-4.00 /KO.OOOl 


Non-methadone outpatient with 90-1 19 days of 
intended treatment versus correctional with 90-1 19 
days of intended treatment 


42.4% versus 51.6% 

A^with 1 DF = 3.14 
n.s. 


1 15.5 days versus 1 10.5 days 
/ with 471 DF = 0.58, n.s. 


Non-methadone outpatient with 120 days or more of 
intended treatment versus long-term residential with 
120 days or more of intended treatment versus 
correctional with 120 or more days of intended 
treatment 


27.8% versus 39.4% versus 
22.0% 

with 2 DF = 35.87 

p<0.001 


121.2 days versus 102.2 days 
/with 1607 DF = 4.02 /KO.OOOl 

102.2 days versus 93.4 days 
/ with 930 DF = 1 .46, n.s. 

121 .2 days versus 93.4 days 

/ with 1 1 57 DF = 4.09 /KO.OOOl 


Short-term residential with 21-30 days of intended 
treatment versus correctional 21-30 days of intended 
treatment 


84.4% versus 88.9% 

with 1 DF = 2.44 
n.s. 


28.2 days versus 28 days 
/ with 660 DF = 0. 1 6, n.s. 


Short-term residential with 21-30 days of intended 
treatment versus long-term residential with 120 or 
more days of intended treatment 


84.4% versus 39.4% 

A^with 1 DF = 226.16 
/KO.OOl 


28.2 days versus 102.2 days 
t with 834 DF =2 1 .68 p<0.0001 
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residential cell (p<0.0001). However, the long-term residential cell and the correctional cell did 
not differ significantly in terms of length of stay. 

The results of the final contrast, the comparison of the two residential analysis cells, 
indicated that the short-term residential cell exhibited a significantly higher completion rate 
relative to the long-term residential cell (p<0.001). These two cells also differed significantly in 
terms of LOS with the long-term residential cell exhibiting a significantly higher mean LOS 

(p<0.0001). 

2. LOGISTIC REGRESSION 

Exhibits III-4, III-5, and III-6 present the results of the within-cell logistic regression 
models. Client’s recollection of seeing their treatment plans yielded a positive and significant 
parameter estimate in all of the models in which it was included. Providing vocational training 
was also positive in several cells — the short-term residential cell and the 120 days or more non- 
methadone and long-term residential cells — but was significant only within the two long-term 
cells. The use of staff specifically designated as case managers yielded positive parameter 
estimates in both of the long-term cell models in which it was included but was significant only 
within the non-methadone outpatient cell. The correctional modality cells in the 21-30 day and 
41-89 day intended length of treatment strata are not represented in Exhibit III-6 because no 
significant zero-order correlations were found within these cells between the dependent variable 
and any of the SDU-level independent variables considered in the analysis. The one SDU-level 
variable included in the logistic regression model for the 90-119 day correctional analysis cell, 
‘percentage of staff of ethnic or racial minority’, yielded a positive but nonsignificant parameter 
estimate. 

Overall, the model for the non-methadone outpatient cell in the 120 days or more stratum 
had the most significant predictors. Within this cell, having case managers and providing 
services tailored to populations defined by specific characteristics (e.g., homeless, pregnant, etc.) 
significantly increased the odds of treatment completion. The only SDU characteristic which 
significantly decreased the odds of treatment completion was patients requesting revisions to 
their treatment plans as a way in which clients participate in the development of their plans. 

This finding warrants further investigation. 

Model improvement tests using -2LL differences showed that all of the models with 
background variables plus independent variables represented significant improvements over the 
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models containing only background variables (p<0.005). All of the models including 
background and independent variables exhibited higher adjusted generalized coefficients of 
determination and higher percent correct and percent concordant values. In addition, false 

positive and false negative rates were lower for these models. The greatest increase in was 
observed for the model with the background + independent variables in the long-term residential 
cell. 



Whether the client reported seeing their treatment plan is by far the most predominant 
predictor of treatment completion. This independent variable was found to be significant in all of 
the models in which it was included. Taken at face value, this pattern might suggest that 
programs can increase their completion rates by raising the visibility of the treatment planning 
process to the client. Since this is something that programs clearly have the ability to do, it 
potentially represents a very significant finding. Before drawing this conclusion, however, 
plausible alternatives must be considered and, if possible, investigated. 

We considered two alternative explanations for the effect of the patient saw treatment 
plan variable. First, there is the possibility that completers were more likely to report seeing their 
plans because they simply stayed longer. Treatment plans are often developed over a lengthy 
period, often as long as 90 days after admission (CSAT, 1994). Therefore, non-completers may 
have been less likely to see their treatment plan simply because they were more likely to leave 
before it was completed. To investigate this possible explanation, we first compared the 
distributions of completers and non-completers on length of stay within cells. While completers 
in each analysis cell stayed longer on average (p<0.05 for all cells), there was tremendous 
overlap across the two distributions. For example, 13 noncompleters in the 120 days or more 
long-term residential cell stayed longer than 240 days and 3 clients in the 90-119 day non- 
methadone outpatient cell “completed” in one day. Thus, this is not a simple case of non- 
completers routinely leaving earlier than completers. Next, we looked at the relationship 
between length of stay (LOS) and seeing a treatment plan. In each of the 5 cells where we found 
the effect of seeing a treatment plan on completion, seeing a plan was positively correlated with 
LOS as expected. However, these correlations were rather weak and differed based on 
completion status, see Exhibit III-7. If the relationship between the clients’ recollections of 
having seen a treatment plan and completion was simply an artifact of the relationship between 
seeing the plan and LOS, the correlations should be observed for both completers and non- 
completers. Because the range of lengths of stay is similar across these two groups of clients for 
all analysis cells except short-term residential, the observation that completers who stay longer 
are not more likely to report having seen a treatment plan suggests that what is driving the effect 
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of the treatment plan variable is not merely a simple relationship between seeing the treatment 
plan and LOS. 



Exhibit IIT-7 

Correlations Between Patient Saw Treatment Plan Variable and 
Length of Stay Within Analysis Cells 


analysis cell 


Those who did and did not complete 
treatment considered togethef:' * 


Those who did and did not complete 
treatment considered separately: 


non-methadone outpatient, 90- 
1 19 days intended length of 
treatment 


r = 0.15 p<0.021 


completers r = -0.03 p<0.701 

non-completers r = 0.21 p<0.021 


non-methadone outpatient, 120+ 
days intended length of 
treatment 


r = 0.19 p<0.0001 


completers r = 0.04 p<0.594 

non-completers r = 0.21 p<0.001 


short-term residential, intended 
length of treatment 21-30 days 


r = 0.16 p<0.0012 


completers r = 0. 1 4 p<0.007 

non-completers r = 0.04 p<0.774 


long-term residential, 120+ days 
intended length of treatment 


r = 0.35 p<0.0001 


completers r = 0.03 p<0.593 

non-completers r = 0.36 p<0.001 


correctional facility, 120+ days 
intended length of treatment 


r = 0.27 p<0.000\ 


completers r = 0.0 1 p<0.999 

non-completers r = 0.29 p<0.001 



The relationship between the SDU-level variable “time treatment plan developed” (B236) 
and client saw treatment plan was also investigated. For those clients for which this data was 
available, (all cells except short-term residential were missing 60 percent or more values for this 
variable) we found that most of the clients who had lengths of stay of 14 days or less and who 
were from SDUs that reported they usually develop treatment plans during the first 14 days after 
admission, responded that they had not seen a treatment plan. This pattern was consistent across 
all of the analysis cells exhibiting the client saw treatment plan effect with the exception of the 
short-term residential group. Furthermore, there are clients in this group in most cells who 
completed within a 14-day period and reported seeing a treatment plan as well as non -completers 
who stayed in treatment beyond this point who reported not seeing a treatment plan. This is 
inconsistent with what would be expected if the ‘time treatment plan was developed’ variable 
was responsible for the effect of seeing the treatment plan. 

In order to determine the effect of the relationship between LOS and clients’ recollections 
of having seen a treatment plan on the logistic regression results, LOS was added to the models 
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for all cells. Including this variable had the effect of reducing the parameter estimates for the 
client saw plan variable in two of the analysis cells — ^the 120 days or more non-methadone 
outpatient and correctional cells. In both of these cases, the odds ratios were positive (ORs=I .46, 
1 .74) but not significant. However, all other significant independent variables in the models 
described in Exhibits III-4, III-5, and III-6 remained significant after the inclusion of LOS. 

The second alternative explanation considered is that the ‘patient saw treatment plan’ 
finding could represent a reporting bias. That is, non-completers may have been less likely to 
recall seeing a treatment plan because they were less engaged in the treatment process generally. 
The data do not permit any investigation of this hypothesis, however. 

Exhibit III-8 presents the results of the sensitivity analysis in which the non-completers in 
the models were restricted to those clients who were classified as having had definite negative 
exits from their programs. These data show that the within-cell regression results were not 
significantly affected by including those clients who were classified as having been transferred to 
another program, incarcerated, or as “other reason(s) for discharge.” 
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Exhibit III-8 

Sensitivity Check of Within-Cell Logistic Regression Models 


INTENDED 
LENGTH OF 
TREATMENT 


NON-METHADONE OUTPATIENT 


90-119 days 


Independent variables added to the model include: 
average length of individual counseling sessions OR=0.97 
patient saw treatment plan OR=2.96 (1.16,7.53)* 


120 days or 
more 


Independent variables added to the model include: 

% staff in recovery 

SDU matched patients to providers OR=1.03 
SDU has case manager(s) OR=2.79 (1.60, 4.85)** 

SDU tailors to populations OR=3.76 (1.47,9.57)** 

SDU provides vocational training OR=2.73 (1.07,6.98)* 
patient asks for changes to tx plan OR=0.37 (0.22,0.61)* 
patient saw treatment plan OR=2.24 ( 1 .47,3.42)** 


SHORT-TERM RESIDENTIAL 


21-30 days 


Independent variables added to the model include: 
average length of individual counseling sessions OR=0.95 
SDU provides vocational training OR=9.91 
patient saw treatment plan OR=4.73 (2.08,10.74)** 


LONG-TERM RESIDENTIAL 


120 days or 
more 


Independent variables added to the model include: 

SDU has case manager(s) OR=1.10 

SDU provides vocational training OR=2.40 (1 .34,4.28)** 

patient saw treatment plan OR=9.68 (4.88,19.22)*** 


CORRECTIONAL FACILITY 


90-1 19 days 


Independent variables added to the model include: 
% of staff in recovery OR=2.58 


1 20 days or 
more 


Independent variables added to the model include: 
patient saw treatment plan OR=2.98 



* Odds ratios significant at p<0.05 
** Odds ratios significant at p<0.005 
*** Odds ratios significant at p<0.001 
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IV. Summary and Conclusions 



This section contains a statement of potential limitations of the analyses reported in this 
study. It also contains a discussion of the results of the analyses and implications for further 
research, policy, and treatment practice. 

1. LIMITATIONS 

Prior to the interpretation of the results, certain potential limitations should be considered. 
Intraclass correlation of observations (within SDU) were not addressed. First, because the 
analysis did not take this measure of homogeneity into account, variance terms may have been 
underestimated and the significance levels of the independent variables may thus be inflated. 
Second, SDUs differed considerably in their formal criteria for successful completion (e.g., client 
follows treatment plan, remains clean for a length of time, consistent attendance at follow-up 
meetings, changes in lifestyle, etc.). The influence of this factor on the results of the analysis is 
unknown. In addition, interactions between client-level and program-level factors were not 
investigated. 

2. CONCLUSIONS 

Notwithstanding these potential limitations, the descriptive comparisons of completion 
rates and median actual lengths of stay for the 8 analysis cells revealed some interesting findings. 
These results include: 

■ In general, shorter programs have higher treatment episode completion rates but 
longer programs retain clients for longer periods. 

■ The median actual lengths of stay for clients in programs with intended lengths of 
treatment of 120 days or more were lower than in the 90-1 19 day programs. This 
finding suggests that setting the intended length of treatment at 120 days or more may 
be counterproductive with respect to retaining clients. 

■ Among long-term programs (120 days or more intended length of treatment), 
residential programs exhibit higher completion rates than programs in non-methadone 
outpatient or correctional modalities. 

■ The results of the logistic regression analysis show that SDU-level factors clearly 
increase the odds of program completion and that the inclusion of SDU-level 
variables improved model fit in all the cells considered. 
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■ The predominance of the effect of the ‘patient saw treatment plan’ variable across 
analysis cells suggests that engaging the client in the treatment plan development 
process may be an important determinant of treatment completion. 

■ Results from the 120 days or more non-methadone outpatient and long-term 
residential cells suggest that providing vocational training, at least within the context 
of a long-term program, may be another important determinant of treatment 
completion. Attending educational and vocational classes while in treatment has been 
found to be significantly related to retention. Simpson et al. (1997) reported that in 
the Drug Abuse Treatment Outcome Study (DATOS), the odds of staying in treatment 
for 90 days or longer in long-term residential programs increased sixfold if the client 
was enrolled in such classes. Thus, providing vocational training, either on- or off- 
site, in programs with long intended durations may be one way that programs could 
modify their service delivery to increase retention and thus the odds of treatment 
completion. 

■ Results from the 120 days or more non-methadone outpatient cell suggest that having 
staff specifically designated as case managers may be another determinant of 
treatment completion. Case management has been described as a process that focuses 
on the whole individual, “stresses comprehensive assessment, service planning, and 
service coordination, and advocacy to address multiple aspects of a client’s life,” 
(CSAT 1997). Research suggests that case management’s effectiveness as an adjunct 
to substance abuse treatment is tied to patient retention (CSAT 1997). Case 
management’s principal role is to keep clients engaged in treatment and moving 
toward recovery — in a sense, to facilitate retention (CSAT 1997). Given this, we 
might expect that the effect of having a case manager would have been detected in 
more than one analysis cell. One possible explanation for why the case manager 
effect was not stronger relates to the way in which the NB AR survey item was 
worded. The question merely asks if the SDU had a staff member who was 
specifically designated as a case manager. Absence of a case manager does not 
necessarily mean the absence of the provision of case management services. Nor does 
the presence of a case manager mean that all clients received case management 
services. Furthermore, the presence of such a staff member tells us nothing about the 
intensity of the services provided. Overall, this NBAR variable does not permit an 
optimal test of whether case management increases retention in substance abuse 
treatment programs. 

■ Results from this cell also indicate that having services tailored to specific 
populations, at least within the context of this type of program, may also be a 
predictor of treatment completion. Orwin et al. (1998), in their analysis of data from 
the NIAAA Cooperative Agreement Program, pointed out that increased 
responsiveness to client needs from the perspectives of the clients themselves is one 
factor that may have a positive impact on retention (e.g., increasing the cultural 
sensitivity to certain subgroups of the client population). 
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3. IMPLICATIONS FOR FURTHER RESEARCH, POLICY, AND PRACTICE 



These results imply several avenues for further research: 

■ An investigation of interactions between client level variables and SDU-level 
variables (also among client-level variables and among SDU-level variables) is 
warranted. 

■ Future analyses should look closely at those clients who drop out in order to develop 
profiles of “early” and “late” leavers. Such an investigation may allow for predictions 
to be made and special attention paid to those who may be at risk of leaving the 
program early. 

■ Further work with this data set should also focus on why clients leave programs. 
Exploring the reasons for leaving treatment might provide insight into how programs 
could modify their service delivery to increase retention. 

■ Additional analyses should be carried out to determine whether there are there 
patterns in terms of when people leave their programs. If patterns are detected, 
programs could use this information to try to develop strategies to engage clients at 
critical times. 

■ Investigations that relate treatment episode completion and treatment outcomes 
(reduced substance abuse, etc.). 

Implications for policy, particularly the assignment of priorities and the allocation of resources, 
that can be drawn from this research include the following: 

■ Consideration should be given to developing an intensive moderate-length (about 90 
days) residential program. This program could include the elements now found in 
long-term residential programs but in a more intensive treatment regime. Our 
research shows that clients are more likely to finish the short-term residential program 
than the long-term programs. Furthermore, the average length of stay for the long- 
term residential program is 102.2 days. A moderate-length intensive residential 
program may prove a successful treatment approach. 

■ More attention should be paid on the impact of case management on client outcomes 
for NMO modalities. Is it necessary for a specific person to be designated as a case 
manager, or can a number of individuals assume this role? Is a case manager more 
important for certain types of clients than others? Does case management enhance or 
reduce delivered treatments? 



i' "!l 050\RTENTION\RETENT.WPD 



NEDS, July 19, 1999, Page 25 



Summary and Conclusions 



An implication for treatment practice can be drawn from these results, as well. The positive 
effects of clients seeing their treatment plans needs to be explored. Client involvement in the 
treatment plan can vary. Clients can actively participate in its development and formally agree to 
it, or just review the plan passively. In spite of the ambiguity, allowing clients a role in their 
treatment plans appears to be a low-risk way to improve treatment completion in all modalities. 
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Appendix 



Description of the National Treatment Improvement Evaluation Study 

AND 

Center for Substance Abuse Treatment Demonstrations ( 1990 - 1992 ) 

The National Treatment Improvement Evaluation Study (NTIES) was a national 
evaluation of the effectiveness of substance abuse treatment services delivered in comprehensive 
treatment demonstration programs supported by the Center for Substance Abuse Treatment 
(CSAT). The NTIES project (1992-1997) was designed and performed for CSAT by the National 
Opinion Research Center at the University of Chicago with assistance from Research Triangle 
Institute. The NTIES project collected longitudinal data between FY 1992 and FY 1995 on a 
purposive sample of clients in treatment programs receiving demonstration grant funding from 
CSAT. Client-level data were obtained at treatment intake, at treatment exit, and 12 months after 
treatment exit. Service delivery unit (SDU) administrative and clinician (SDU staff) data were 
obtained at two time points, 1 year apart. 

1. THE NTIES DESIGN 

1.1 The Administrative/Services Component 

The NTIES study design had two levels — an administrative or services component and a 
clinical treatment outcomes component. The administrative component was designed to assess 
how CSAT demonstration funds were used, what improvements in services were implemented at 
the program level, and what kind and how many programs and clients were affected by the 
demonstration awards. Four data collection instruments were used to gather administrative/ 
services data: the NTIES Baseline Administration Report (NEAR), the NTIES Continuing 
Administrative Report (NCAR), the NTIES Exit Log, and the NTIES Clinician Form (NCF). 

The unit of analysis for the administrative component was the SDU, defined by CSAT as 
a single site offering a single level of care. The classification of level of care is based on three 
parameters: 



■ Facility type (e.g., hospital, etc.) 

■ Intensity of care (e.g., 24-hour, etc.) 

■ Type of service (e.g., outpatient, etc.). 

An SDU could be a stand-alone treatment provider, or it could be one component of a multi- 
tiered treatment organization. For example, a large, county mental health agency may be the 
organization within which the SDU is located. The organization may have multiple substance 
abuse treatment components, such as a county hospital and a county (ambulatory) mental health 
center. The county hospital may have multiple SDUs, such as an inpatient detoxification service, 
an outpatient counseling service, and a hospital satellite center providing transitional care. In 
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Description ofNTIES and CSAT Treatment Demonstrations 



summary, the SDU provided NTIES evaluators with a stable, uniform level of comparison for 
examining service delivery issues. 

A range of key clinician-specific data elements (within the administrative component) 
were assessed using the NCF. The NCF items were an important adjunct to the facility- (SDU) 
level instruments; these items assessed clinician training, experience, client exposure, and service 
provision, and were completed by all counseling and clinical (medical and therapeutic) staff at 
the individual SDUs. 

1.2 Clinical Treatment Outcomes Component 

The unit of analysis for the clinical treatment outcomes component was individual client 
data. NTIES measured the clinical outcomes of treatment primeirily through a “before/after” or 
“pre- to post-treatment” design. This method compares behaviors or other individual 
characteristics in the same participants, measured in similar ways, before and after an 
intervention. 

Information about clients’ lives for the before period were obtained from the NTIES 
Research Intake Questionnaire (NRIQ), which was administered sometime during the clients’ 
first 3 weeks of treatment. The specific areas assessed included: 

■ Drug and alcohol use 

■ Employment 

■ Criminal justice involvement and criminal behaviors 

■ Living arrangements 

■ Mental and physical health. 

Information about clients’ lives for the after period were obtained from the NTIES Post- 
discharge Assessment Questionnaire (NPAQ), with the same areas assessed at roughly 12 months 
post-treatment. Other client data sources included a treatment discharge interview (NTIES 
Treatment Experience Questionnaire, NTEQ), abstracted client records, urine drug screens 
collected at the time of the follow-up interview, and arrest reports from state databases. 

1.3 The Outcome Analysis Sample 

Between August 1993 and October 1994, research staff successfully enrolled 6,593 
clients at 71 SDUs to participate in three waves of an in-person, computer-assisted data 
collection protocol. These SDUs were chosen from the universe of treatment units receiving 
demonstration grant funding from CSAT. Some of the selected facilities were wholly supported 
by CSAT awards, while others received only indirect support or none. 
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Clients were interviewed three times: shortly after admission on their first day of 
treatment, when they left treatment, and 12 months after the end of treatment. Less than 10 
percent of the eligible clients refused or avoided participation, and more than 83 percent of the 
recruited individuals (5,388 clients) completed a follow-up interview. Additional sample 
exclusions included: 

■ Missing or undetermined treatment exit date 

■ Inappropriate length of follow-up interval (less than 5 or more than 1 6 months) 

■ Clients incarcerated for most or all of the follow-up period (nearly all had been treated 
while incarcerated, and were not yet released). 

The additional sample exclusions resulted in a final outcome analysis sample of 4,41 1 
individuals. 

2. TREATMENT DEMONSTRATION PROGRAMS 

CSAT initiated three major demonstration programs and made 1 57 multi-year treatment 
enhancement awards across 47 states and several territories during 1990 through 1992. One 
objective common to all demonstrations was CSAT’s emphasis on the provision of 
“comprehensive treatment” services to targeted client populations. The recipients of these 
awards focused special attention on the substance abuse treatment service needs of minority and 
special populations located primarily within large metropolitan areas. The demonstration 
programs are briefly described below. 

2.1 Target Cities 

Under this demonstration, nine metropolitan areas were selected to receive awards, of 
which half were included in the NTIES purposive sample. The following treatment improvement 
activities were explicitly provided for in the awards: 

■ Establishment of a Central Intake Unit (CIU) with automated client tracking and 
referral systems in place 

■ Provision of comprehensive services, including vocational, educational, biological, 
psychological, informational, and lifestyle components 

■ Improved inter-agency coordination (e.g., mental health, criminal justice, and human 
service agencies) 

■ Services for special populations — adolescents, pregnant and postpartum women, 
racial and ethnic minorities, and public housing residents. 
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2.2 Critical Populations 

Under this demonstration program, awardees were required to implement “model 
enhancements” to existing treatment services for one or more of the following critical 
populations: racial and ethnic minorities, residents of public housing, and/or adolescents. 

Special emphasis was given to services provided to the homeless, the dually diagnosed, or 
persons living in rural areas. A total of 130 grants were awarded, covering services such as 
vocational support/counseling, housing assistance, integrated mental health and/or medical 
services, coordinated social services, culturally directed services, and others. 

2.3 Incarcerated and Non-Incarcerated Criminal Justice Populations 

Under this demonstration program, funds were directed toward improving the standard of 
comprehensive treatment services for criminally involved clients in correctional and other 
settings. Some program emphasis was placed on ethnic and/or racial minorities. Nine 
correctional setting demonstrations were funded: five in prisons, three in local jails, and one 
across a network of juvenile detention facilities. All projects included a screening component to 
identify substance-abusing inmates, a variety of targeted treatment interventions (e.g., therapeutic 
communities, intensive day treatment programs), and a substantial aftercare component. 

A total of 10 non-incarcerated projects were funded. Five programs targeted 
interventions at clients in diversionary programs, three focused services on probationers or 
parolees, and two programs targeted both populations. Almost all of the funded demonstration 
projects included the following components: 

■ Basic eligibility determination, followed by systematic screening and assessment 

■ Referral to treatment 

■ Graduated sanctions and incentives while in treatment 

■ Intensive supervision in treatment 

■ Community-based aftercare with supervision and service coordination. 

In total, 19 criminal justice projects were funded as part of the CSAT 1990-1992 demonstrations, 
and as indicated in the next section, these projects were purposively over-sampled in order to 
obtain a more robust evaluation of this program. 
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3. DESCRIPTION OF SDUS AND CLIENTS BY TREATMENT MODALITY AND 
PROGRAM TYPE 

The 71 SDUs contributing clients to the outcome analysis sample are characterized by 
modality and (demonstration) program type in Exhibit A-1 below. Among the 698 SDUs in the 
NTIES universe: 52 percent (n=365) were Target Cities programs, 39 percent (n=274) were 
Critical Populations programs, and 9 percent (n=59) were Criminal Justice programs. 

In terms of the SDUs sampled for the NTIES outcome analysis, 44 percent were Target 
Cities programs, 38 percent were Critical Populations programs, and 23 percent were Criminal 
Justice programs. Criminal Justice SDUs were purposely over-sampled as part of the NTIES 
evaluation design (CSAT, 1997). Nearly half of the sampled SDUs were (non-methadone) 
outpatient programs, and about one-quarter were long-term residential programs. 

As shown in Exhibit A-2, 59 percent of all NTIES clients were sampled from Target Cities 
SDUs. Slightly over 21 percent of all NTIES clients were sampled from Critical Populations 
SDUs, and 20 percent were sampled from Criminal Justice SDUs. Outpatient (non-methadone) 
SDUs treated over one-third (35%) of the clients in the outcomes analysis sample, and almost 80 
percent of these were sampled from Target Cities programs. 

Readers who are interested in more detailed information about the NTIES project are 
invited to visit the NEDS Web site at: http://neds.calib.com. The NEDS Web site provides the 
full-length version of the NTIES Final Report (1997), as well as copies of all data collection 
instruments employed in NTIES. 
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EXfflBIT A-1 

SDUs IN THE Outcome Analysis Sample 


Program Title 
Number of SDUs 
(% of NTIES 
Universe)^ 


NTIES 

Sample 


Methadone 


Outpatient 


Long-Term 

Residential 


Short-Term 

Residential 


Correctional 


Target Cities 

n=365 (52%) 


31 

(44%) 


6 


15 


6 


4 


0 


Critical 

Populations 

n=274 (39%) 


27 

(38%) 


1 


13 


10 


3 


0 


Criminal Justice 

n=59 (9%) 


13 

(23%) 


0 


5 


0 


0 


8 


Totals 

N=698 (100%) 


71 

(100%) 


7 


33 


16 


7 


8 



Exhibit A-2 

Distribution of Clients in the Outcomes Analysis Sample 


Program Title 
Number of Clients 
(% of Analysis Sample) 


Methadone 


Outpatient 


Long-Term 

Residential 


Short-Term 

Residential 


Correctional 


Target Cities 


377 


1,214 


504 


505 


0 


n=2,600 (59%) 


(89%) 


(78%) 


(60%) 


(58%) 




Critical Populations 


45 


220 


298 


368 


0 


n=931 (21%) 


(11%) 


(14%) 


(35%) 


(42%) 




Criminal Justice 


0 


132 


39 


0 


709 


n=880 (20%) 




(8%) 


(5%) 




(100%) 


Totals 












n=4,411 (100%) 


422 


1,566 


841 


873 


709 



BEST COPY AVAILABLE 



The original NTIES universe of SDUs included a program type called Specialized Services, 
Because clients for the outcome analysis sample were not drawn from these SDUs (n=94), they 
are excluded from the Exhibit. 
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